A primal-dual algorithm with optimal stepsizes and its application in decentralized consensus optimization

We consider a primal-dual algorithm for minimizing f(x) + h(Ax) with differentiable f . The primal-dual algorithm has two names in literature: Primal-Dual Fixed-Point algorithm based on the Proximity Operator (PDFPO) and Proximal Alternating Predictor-Corrector (PAPC). In this paper, we extend it to solve f(x) + h l(Ax) with differentiable l and prove its convergence under a weak condition (i.e., under a large dual stepsize). With additional assumptions, we show its linear convergence. In addition, we show that this condition is optimal and can not be weaken. This result recovers the recent proposed positive-indefinite linearized augmented Lagrangian method. Then we consider the application of this primal-dual algorithm in decentralized consensus optimization. We show that EXact firsT-ordeR Algorithm (EXTRA) and Proximal Gradient-EXTRA (PG-EXTRA) can be consider as the primal-dual algorithm applied on a problem in the form of h l(Ax). Then, the optimal upper bound of the stepsize for EXTRA/PG-EXTRA is derived. It is larger than the existing work on EXTRA/PG-EXTRA. Furthermore, for the case with strongly convex functions, we proved linear convergence under the same condition for the stepsize.